User interface having changeable topography

ABSTRACT

A user interface having changeable topography is disclosed. The user interface can have a shape changeable surface that can selectively alter according to an input so as to provide changeable topography of the user interface. The surface can include individual nodes that can raise above or lower below the initial surface. Alternatively, the surface can include a shape changeable material that can change the shape of portions thereof into discrete shapes above or below the initial surface. Alternatively, the surface can include a deformable material that can deform portions thereof into discrete forms above or below the initial surface. The changeable topography can define different user interface layouts. The user interface can, for example, be associated with input and/or output devices, such as touch pads, touch screens, and the like.

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/409,391, filed Mar. 23, 2009 and published on Jun. 24, 2010 as U.S.Patent Publication No. 2010/0162109, which claims priority to U.S.Provisional Patent Application No. 61/140,050, filed Dec. 22, 2008; thecontents of which are herein incorporated by references in theirentirety for all intended purposes.

FIELD OF THE INVENTION

This relates to user interfaces and, more particularly, to userinterfaces that can change topography.

BACKGROUND OF THE INVENTION

Many types of input devices are presently available for performingoperations in a computing system, such as buttons or keys, mice,trackballs, touch sensor panels, joysticks, touch pads, touch screensand the like. Touch screens and touch pads, in particular, are becomingincreasingly popular because of their ease and versatility of operationas well as their declining price. Touch screens can include a touchsensor panel, which can be a clear panel with a touch sensitive surface,and a display device that can be positioned behind the panel so that thetouch sensitive surface can substantially cover the viewable area of thedisplay device. Touch screens can allow a user to perform variousfunctions by touching the touch sensor panel using one or more fingers,a stylus or other object at a location dictated by a user interface (UI)comprising virtual buttons, keys, bars, displays, and other elements,being displayed by the display device. In general, touch screens canrecognize a touch event and the position of the touch event on the touchsensor panel, and the computing system can then interpret the touchevent in accordance with the display appearing at the time of the touchevent, and thereafter can perform one or more actions based on the touchevent.

Similarly, touch pads can include a touch sensor panel having a touchsensitive surface. Touch pads can allow a user to perform variousfunctions by touching the touch sensor panel using one or more fingers,a stylus or other object at a location dictated by a UI comprising atouch space. In general, touch pads can recognize a touch event and theposition of the touch event on the touch sensor panel, and the computingsystem can then interpret the touch event in accordance with theposition of the touch event, and thereafter can perform one or moreactions based on the touch event.

Touch screens and touch pads can typically have a smooth fixed outersurface through which inputs/outputs can be made. For example, the outersurface can act as an input mechanism that receives touch inputs such astaps, slides and other gestures. With touch screens, the outer surfacecan further act as a window for displaying text and graphics that changeduring use. In most cases, the physical surfaces of these devices can besmooth and fixed. In some cases, they can be flat and planar. Theygenerally do not include tactile features like buttons and therefore canbe used across many modes, applications or platforms. However, becausethese surfaces generally do not include tactile features, the user canhave a difficult time locating UI elements without looking at the visualscreen to ensure that the desired location is touched.

SUMMARY OF THE INVENTION

This relates to a user interface that can physically change topographyto create different tactile configurations at the user interfacesurface. In some embodiments, the user interface can change topographyaccording to a desired user interface state. The user interface statecan, for example, be based on a mode of an electronic device in whichthe user interface is used and/or a particular preference of a user. Insome embodiments, the user interface can change topography according toan event, such as a touch event on the user interface surface. Thechanging topography can define different user interface layoutsaccording to the needs of device.

In some embodiments, the user interface can include a shape changeablesurface configured to selectively alter topography of the user interfaceso as to provide a variable tactile feel of the user interface. Theshape changeable surface can include individual nodes that can be raisedabove or lowered below the initial surface. Alternatively, the shapechangeable surface can include shape changeable material that can changeshape to form discrete shapes above or below the initial surface.Alternatively, the shape changeable surface can include deformablematerial that can deform into discrete forms above or below the initialsurface.

In some embodiments, the user interface can include shape changeablenodes proximate to the user interface surface and configured toselectively alter so as to alter a proximate region of the surface inorder to alter topography of the user interface. The nodes can includeelectromechanical devices that can move to push against or pull awayfrom the surface. Alternatively, the nodes can include shape changeableelements that can change shape to push against or pull away from thesurface. Alternatively, the nodes can include deformable elements thatcan deform to push against or pull away from the surface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary user interface that can changetopography according to embodiments of the invention.

FIG. 2 illustrates an exemplary user interface that can changetopography by selectively altering individual shape changeable nodesaccording to embodiments of the invention.

FIG. 3 illustrates an exemplary user interface that can changetopography by selectively altering a group of shape changeable nodesaccording to embodiments of the invention.

FIG. 4 illustrates an exemplary user interface that can changetopography using various modules according to embodiments of theinvention.

FIG. 5 illustrates an exemplary method for changing the topography of auser interface according to embodiments of the invention.

FIG. 6 illustrates an exemplary touch screen having a user interfacethat can change topography according to embodiments of the invention.

FIG. 7 illustrates an exemplary touch screen having a user interfacethat can change topography to form virtual buttons according toembodiments of the invention.

FIG. 8 illustrates an exemplary touch screen having a user interfacethat can change topography to form a virtual keypad according toembodiments of the invention.

FIG. 9 illustrates an exemplary touch screen of an electronic devicehaving a user interface that can change topography according toembodiments of the invention.

FIG. 10 illustrates an exemplary user interface that can changetopography using electromechanical devices according to embodiments ofthe invention.

FIG. 11 illustrates an exemplary user interface that can changetopography using electromechanical devices to raise portions of the userinterface surface according to embodiments of the invention.

FIG. 12 illustrates an exemplary user interface that can changetopography using electromechanical devices to lower portions of the userinterface surface according to embodiments of the invention.

FIG. 13 illustrates an exemplary circuit for changing topography of auser interface using electromechanical devices according to embodimentsof the invention.

FIG. 14 illustrates an exemplary electromechanical device that can beused for changing topography of a user interface according toembodiments of the invention.

FIG. 15 illustrates an exemplary user interface of a touch screen thatcan change topography according to embodiments of the invention.

FIG. 16 illustrates an exemplary user interface of a touch screen thatcan change topography to form a virtual button according to embodimentsof the invention.

FIG. 17 illustrates an exemplary method for changing the topography of auser interface of a touch screen according to embodiments of theinvention.

FIG. 18 illustrates an exemplary touch sensing device having a userinterface that can change topography according to embodiments of theinvention.

FIG. 19 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel according toembodiments of the invention.

FIG. 20 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a keypad according toembodiments of the invention.

FIG. 21 illustrates an exemplary touch sensing device having a userinterface that can change topography to form push buttons according toembodiments of the invention.

FIG. 22 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel according toembodiments of the invention.

FIG. 23 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel and pushbuttons according to embodiments of the invention.

FIG. 24 illustrates an exemplary user interface that can changetopography using shape changeable nodes according to embodiments of theinvention.

FIG. 25 illustrates an exemplary user interface that can changetopography using shape changeable nodes to raise portions of the userinterface surface according to embodiments of the invention.

FIG. 26 illustrates an exemplary user interface that can changetopography using shape changeable nodes to lower portions of the userinterface surface according to embodiments of the invention.

FIG. 27 illustrates an exemplary circuit for changing topography of auser interface using shape changeable nodes according to embodiments ofthe invention.

FIG. 28 illustrates an exemplary shape changeable node that can elongateor shrink for changing topography of a user interface according toembodiments of the invention.

FIG. 29 illustrates an exemplary shape changeable node that can rotatefor changing topography of a user interface according to embodiments ofthe invention.

FIG. 30 illustrates an exemplary user interface that can changetopography having a sensing device integrated with the user interfaceaccording to embodiments of the invention.

FIG. 31 illustrates an exemplary user interface that can changetopography having a sensing device applied to a surface of the userinterface according to embodiments of the invention.

FIG. 32 illustrates an exemplary user interface that can changetopography having a sensing device separate from the user interfaceaccording to embodiments of the invention.

FIG. 33 illustrates an exemplary user interface that can changetopography by selectively altering shape changeable nodes according toembodiments of the invention.

FIG. 34 illustrates an exemplary user interface that can changetopography by selectively pressing on shape changeable nodes accordingto embodiments of the invention.

FIG. 35 illustrates an exemplary user interface that can changetopography by selectively altering shape changeable nodes in response topressure thereon according to embodiments of the invention.

FIG. 36 illustrates an exemplary touch pad having a user interface thatcan change topography and selectively activate touch regions associatedwith the change according to embodiments of the invention.

FIG. 37 illustrates an exemplary user interface of a touch pad that canchange topography based on a sensed touch according to embodiments ofthe invention.

FIG. 38 illustrates an exemplary user interface of a touch pad that canchange topography to form a scroll wheel based on a sensed touchaccording to embodiments of the invention.

FIG. 39 illustrates an exemplary method of changing the topography of auser interface of a touch pad according to embodiments of the invention.

FIG. 40 illustrates an exemplary user interface that can changetopography using a shape changeable membrane according to embodiments ofthe invention.

FIG. 41 illustrates an exemplary user interface that can changetopography using a shape changeable membrane to raise portions of theuser interface surface according to embodiments of the invention.

FIG. 42 illustrates an exemplary user interface that can changetopography using a shape changeable membrane to lower portions of theuser interface surface according to embodiments of the invention.

FIG. 43 illustrates an exemplary circuit for changing topography of auser interface using a shape changeable membrane according toembodiments of the invention.

FIG. 44 illustrates an exemplary user interface that can changetopography based on a location of a touch event according to embodimentsof the invention.

FIG. 45 illustrates an exemplary user interface changing topography at alocation of a touch event according to embodiments of the invention.

FIG. 46 illustrates an exemplary user interface changing topography at alocation resulting from a horizontal scroll touch event according toembodiments of the invention.

FIG. 47 illustrates an exemplary user interface changing topography at alocation resulting from a vertical scroll touch event according toembodiments of the invention.

FIG. 48 illustrates an exemplary user interface changing topography atmultiple locations based on multiple touch events according toembodiments of the invention.

FIG. 49 illustrates an exemplary method for changing the topography of auser interface based on a location of a touch event according toembodiments of the invention.

FIG. 50 illustrates an exemplary display device having a user interfacethat can change topography according to embodiments of the invention.

FIG. 51 illustrates an exemplary display device having a user interfacethat can change topography by raising a display screen according toembodiments of the invention.

FIG. 52 illustrates an exemplary user interface that can changetopography according to embodiments of the invention.

FIG. 53 illustrates an exemplary computing system having a userinterface that can change topography according to embodiments of theinvention.

FIG. 54 illustrates an exemplary mobile telephone having a userinterface that can change topography according to embodiments of theinvention.

FIG. 55 illustrates an exemplary digital media player having a userinterface that can change topography according to embodiments of theinvention.

FIG. 56 illustrates an exemplary personal computer having a userinterface that can change topography according to embodiments of theinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description of preferred embodiments, reference is madeto the accompanying drawings in which it is shown by way of illustrationspecific embodiments in which the invention can be practiced. It is tobe understood that other embodiments can be used and structural changescan be made without departing from the scope of the embodiments of thisinvention.

This relates to a user interface that can physically change topographyto create different tactile configurations at the surface. In someembodiments, the user interface can change topography according to adesired user interface state. The user interface state can, for example,be based on a mode of an electronic device in which the user interfaceis used and/or a particular preference of a user. In some embodiments,the user interface can change topography according to an event, such asa touch event on the user interface surface. The changing topography candefine different user interface layouts. The user interface can, forexample, be associated with input and/or output devices, such as touchpads, touch screens, and the like.

The topographical change can be implemented by a shape changeablesurface that can include one or more alterable nodes. The nodes can bealtered either individually or in combination and be associated withcertain user interface elements. For example, the nodes can be alteredto define or form a physical or virtual button, a key, a navigation pad,a scroll wheel, and the like. In some embodiments, the nodes cancomprise electromechanical devices that can move from one level toanother to form raised and lowered regions of the user interface. Insome embodiments, the nodes can comprise shape changeable material thatcan change shape to form raised and lowered regions of the userinterface. In some embodiments, the nodes can comprise a deformablemembrane that can deform to form raised and lowered regions of the userinterface.

This type of configuration can create a different look as well as createa different tactile feel at the user interface surface. In so doing, theuser interface can inform the user as to what input and/or output ispertinent for a particular use of an electronic device having the userinterface. That is, shape changeable nodes can be moved, shaped, ordeformed above or below the initial user interface surface in order toproduce a surface of varying topography. When used in conjunction with adisplay, the shape changeable surface can provide an effective andunobtrusive way to emphasize or highlight certain display elements tothe user, and the display can provide a way in which outputs can beviewed by the user. When used in conjunction with a touch sensingdevice, the shape changeable surface can provide an effective andunobtrusive way for a user to know where the user interface elementsare, and the touch sensing device can provide a way in which inputs canbe received when the user interface elements are touched. The nodes cancorrespond directly with a user interface element. For example, thenodes and user interface element can be combined to form an enhanceduser interface element. The nodes can also be used indirectly to helpdefine a user interface element. For example, the nodes can help defineborders or boundaries of the user interface element.

Although some embodiments of this invention can be described herein interms of touch sensitive devices with user interfaces, it should beunderstood that embodiments of this invention are not so limited, butare generally applicable to any devices utilizing other types of sensingtechnologies with user interfaces.

It should be noted that the terms “activate,” “actuate,” “stimulate,”and the like herein can generally be used synonymously to refer to theshape changeable actions of the user interface to change topographyaccording to embodiments of the invention.

FIG. 1 illustrates an exemplary user interface that can changetopography according to embodiments of the invention. In the example ofFIG. 1, user interface 10 can include shape changeable surface 11 havingshape changeable nodes 12 to change the topography of the userinterface. A user interface can be defined as a component associatedwith a device through which a user interacts with the device to provideinput and/or receive output. A topographical (or shape) change can bedefined as a change in height, width, length, orientation,configuration, layout, texture, pattern, three-dimensional form, and thelike of all or a portion of the surface associated with the userinterface. The change can be implemented by physically alteringindividual shape changeable nodes, as in FIG. 2 described below, or byphysically altering a group of shape changeable nodes, as in FIG. 3described below. The nodes can include an actuator that can changebetween physical states, thus causing the nodes to change. For example,the actuator can include a moving member that can move the node from aninitial state to a raised or lowered state or that can deform, ratherthan move, the surface in order to create a shape change. In so doing,the topography of the user interface can be selectively changed.

In some embodiments, the user interface 10 can change topography inconjunction with a user interface state, thereby providing differentphysical layouts of the user interface. For example, the user interfacecan change its topography to have a certain physical layout when usedfor a phone mode and can change its topography to have another certainphysical layout when used for a media player mode. As should beappreciated, many different physical layouts can be used for any numberof modes of a device. In addition, a particular topography can remainstatic during a particular mode to provide a particular tactile feel.Alternatively, the topography can dynamically change during a particularmode to provide a variety of tactile feels.

In some embodiments, rather than change in conjunction with a userinterface state, the user interface 10 can change topography inconjunction with a user input, such as a touch, near touch, or pressevent. For example, the location proximate to the touch can be altered,in some cases continuously, during a touch event. For example, as a userslides a finger across the surface or presses a finger down on thesurface, the node closest to the finger can alter while the remainingnodes do not. This can be implemented with a touch sensing device. Forexample, a controller can monitor the touch sensing nodes and alter theshape changeable nodes based on the location of the nodes that detect atouch, near touch, or press.

Rows and columns or matrices of shape changeable nodes 12 can cooperateto change topography of a user interface. For example, individual nodescan be altered to create distinct pixel shape changes at the userinterface surface. Alternatively, nodes can be altered together to forma larger area or a group of pixel shape changes at the user interfacesurface.

Various components, e.g., a changeable surface and/or a plurality ofchangeable parts, can comprise the shape changeable nodes of the userinterface according to embodiments of the invention. For example, thesurface of the shape changeable nodes can comprise individual nodes thatcan be altered individually by underlying movable or deformable parts.Alternatively, the surface of the shape changeable nodes can comprise aflexible membrane that can be stretched and retracted at designated nodelocations by underlying movable or deformable parts. This can keep thenodes hidden from view until they can be activated, for example, if theflexible membrane is substantially opaque. Alternatively, the surface ofthe shape changeable nodes can comprise a shape changeable membrane thatcan expand and contract when stimulated by an electrical or magneticsignal or by chemical stimulus.

Movable or deformable parts can underlie the node surface. For example,electromechanical devices, e.g., micro actuators, microelectromechanicaldevices, or piezoelectronics, can have movable physical components thatmove up and down when stimulated by a mechanical force or an electricalor magnetic signal, thereby moving the overlying surface. Theelectromechanical devices can be placed adjacent to individual nodes orto certain locations of the flexible membrane. Alternatively, shapechangeable material, e.g., shape memory material, magnetic material, orelectrostatic material, can expand or contract when stimulated by anelectrical or magnetic signal or chemical stimulus, thereby moving theoverlying surface. The shape changeable material can be placed adjacentto individual nodes or to certain locations of the flexible membrane.

The shape changeable nodes will be described in more detail hereinlater.

In the example of FIG. 2, user interface 20 can include shape changeablesurface 21 having shape changeable nodes 22. Shape changeable nodes 22-bcan be individually selectively altered to raise regions at the shapechangeable surface 21, while shape changeable nodes 22-a can remainunaltered in the original surface. In the example of FIG. 3, userinterface 30 can include shape changeable surface 31 having shapechangeable nodes 32. Inner shape changeable nodes 32-b can becollectively selectively altered to raise regions at the shapechangeable surface 31, while outer shape changeable nodes 32-a canremain unaltered in the original surface. In some embodiments, thealtered state can be created by a moving element. In some embodiments,the altered state can be created by a deformable element. In someembodiments, the altered state can be created by a shape changeableelement. All or portions of the surface can be selectively raised and/orlowered to change topography. In one embodiment, all or a portion of thesurface can be raised. In another embodiment, all or a portion of thesurface can be lowered. In yet another embodiment, some portions of thesurface can be raised while other portions can be lowered. By way ofexample, individual button(s) can be formed and/or a wheel, e.g., ascroll wheel with an outer scroll ring and a central button, can beformed in the surface by selectively raising and/or lowering differentportions of the surface.

Although the shape changeable nodes can be described herein in terms ofan orthogonal array of nodes having rows and columns, it should beunderstood that embodiments of this invention are not limited toorthogonal arrays, but can be generally applicable to nodes arranged inany number of dimensions and orientations, including diagonal,concentric circle, three-dimensional, and random orientations.

Although FIGS. 1 through 3 show the user interface as a planar surface,it is to be understood that the surface can embody various forms,including three-dimensional forms. For example, the initial surface canbe contoured to devices having various known shapes, such as a mouse andthe like.

It should be further understood that the shape changeable nodes are notlimited to rectangles, but can take various forms, such as circles,ovals, irregular shapes, and the like, and sizes and can work togetherto form different forms and contoured portions at the surface. Theshapes can be somewhat dependent on the resolution. The nodes can beassociated with points or pixels as well as areas or regions.

A user interface having changeable topography can be associated with atouch sensitive device, such as a touch pad or a touch screen. As such,each shape changeable node of the user interface can be associated withone or more touch sensing nodes of a touch surface of the touchsensitive device. In the case of the touch screen, each shape changeablenode can also be associated with one or more display pixels of the touchscreen display. The resolution of the shape changeable nodes, the touchsensing nodes, and the display pixels can be the same or different. Insome embodiments, the nodes and pixels can have a 1:1:1 relationship inboth size and placement. In other embodiments, the relationship can besomething other than 1:1:1. For example, it can be such that theresolution of the shape changeable nodes can be large compared to thetouch and display pixels, i.e., multiple pixels can be situated withinthe area of a node. The resolution can be dependent on the desired needsof the device. In some embodiments, the touch sensing nodes can bespaced apart and placed between shape changeable nodes. In someembodiments, the touch sensing nodes can cover the entire surfaceincluding portions of the shape changeable nodes.

FIG. 4 illustrates an exemplary user interface that can physicallychange topography using various modules according to embodiments of theinvention. The user interface 40 can for example be associated with theuser interfaces 10, 20, 30 shown in FIGS. 1 through 3 respectively. Theuser interface can be associated with an input/output device that can beoperatively coupled to a host device. The host device can for example bean electronic device, such as a portable computer, or a handheldelectronic device, such as a phone or media player. Generally speaking,the user interface can be applied to a consumer electronic product thatincludes a user interface. In some cases the user interface can beperipherally operable relative to the host, while in other cases theuser interface can be integrated within the host. The user interface canbe particularly useful in portable devices and further handheld devicesthat can have limited real estate for placement of a user interface. Byway of example, the user interface can be integrated within a touchscreen of an iPhone™ or touch pad of an iPod™ manufactured by Apple Inc.of Cupertino, Calif.

In the example of FIG. 4, the user interface 40 can include shapechangeable layer 41 to change topography of the user interface. Theshape changeable layer 41 can for example include a matrix of nodes,each of which can be capable of physically changing a specific locationof the user interface in order to change the topography of the surface.For example, each shape changeable node can be configured to raise orlower the surface of the user interface, thereby changing thetopography. This can be done in with multiple nodes in combination orwith discrete single nodes. That is, the nodes can be controlled in avariety of ways to produce the desired topography, i.e., differentshapes, contours, forms.

The user interface 40 can also include sensing layer 45 to detect atouch or near touch (proximity). Like the shape changeable layer, thesensing layer can include a matrix of nodes. The sensing nodes can beconfigured to detect the presence of an object such as a finger. Thesensing layer 45 can be widely varied. In one example, the sensing layer45 can be based on capacitance. It should be appreciated that othertechnologies including resistive, surface wave, pressure sensing and thelike can be used. In some cases, the sensing layer 45 can even bemultipoint sensing, i.e., capable of sensing multiple objects at thesame time (simultaneously). In one example, the sensing layer 45 can beassociated with touch sensing (or near touch/proximity). The sensingnodes can be configured to cooperate with the shape changeable nodes tochange the topography and sensing areas of the user interface. Forexample, while the shape changeable nodes can alter the shape of theuser interface, the sensing nodes can alter the detectable areas of theuser interface. In one example, the sensing nodes can be activated ordeactivated and the shape changeable nodes can be adjusted based on thedesired user interface.

The user interface 40 can also optionally include display layer 46 todisplay user interface elements as for example when the user interface40 is associated with a touch screen. The display layer 46 can beconfigured to display graphics and text. The display layer 46 can be asingle display or a group of displays that work together. When discrete,the displays can work with the shape changeable nodes to change thetopography of the user interface. For example, the displays can raise orlower. In some cases, the display layer 46 can be unassociated from atouch screen and can work with shape changeable nodes independent of atouch sensor. For example, the display layer can change its topographybased on what is being displayed to create a three-dimensional displayeffect.

The display layer can also be associated with a touch pad having avisual feedback elements as disclosed in U.S. patent application Ser.No. 11/394,493, entitled “Illuminated Touchpad,” published as U.S.Patent Application Publication No. 2007/0152977, and Ser. No.11/591,752, entitled “Touch Pad with Symbols Based on Mode,” publishedas U.S. Patent Application Publication No. 2007/0152983, the contents ofwhich are incorporated herein by reference in their entirety for allpurposes.

The user interface 40 can also include controller module 42 forcontrolling the various layers of the user interface. The controller 42can be, for example, a dedicated processor or a supplemental processor.The controller 42 can even be a part of the host device in somecircumstances. The controller 42 can be in the form of an applicationspecific integrated circuit (ASIC). In some cases, the controller 42 caninclude modules as for example shape module 42-a to controltopographical changes, touch module 42-b to control touch detection, andoptionally display module 42-c to control displays. The modules can bediscrete or integrated, depending on the needs of the user interface.

In some cases, the user interface 40 can also include user interfacestate module 43 that can drive the user interface to change topography,display and/or sensing based on a desired user interface state. The userinterface state module 43 can be part of the user interface 40 orseparate. For example, the user interface state module can be associatedwith the host device. The module 43 can be, for example, software,firmware, or hardware. At various times, the controller 42 can drive theuser interface to change topography. It is to be understood that otherand/or additional components can comprise the user interface accordingto the needs of the user interface.

During operation, the controller 42 can be configured to control thevarious layers 41, 45, 46 in order to produce the desired user interfaceeffect. For example, the shape changeable layer 41 can be controlled toproduce a desired topography or to change the tactile feel of the userinterface during use. In addition, the controller 42 can activate anddeactivate touch regions. For example, the controller 42 can produceactive regions that detect touches and null regions that ignore touches.In addition, the controller 42 can direct the display layer 46 todisplay user interface elements. For example, the controller 42 caninstruct the display layer 46 to display virtual buttons or the like. Itshould be appreciated that the shape points or regions, touch points orregions and display points or regions can be controlled together toproduce the desired user interface effect. In one embodiment, theseelements can be associated with one another. For example, the displaycan display a virtual button, the touch sensor can be active at thevirtual button, and the shape change can create boundaries around thevirtual button or raise and/or lower the virtual button itself. The usercan therefore feel and see the button and touch it to activate an input.

In some embodiments, the controller 42 can adjust each of the layers 41,45, 46 in order to produce layouts based on applications or modes, whichcan be associated with the host. For example, the user interface statemodule 43 can drive the user interface to change to a phone mode or amedia player mode.

In other embodiments, the controller 42 can adjust the layers 41, 45, 46based on how a user interacts with the user interface. For example,after detecting a touch event, the shape changeable nodes associatedwith the touch event can change shape in order to provide feedback tothe user as the user interacts with the user interface. For example, thesurface can change in real time during the touch event. The change canbe based on touch location, acceleration, direction, size, number oftouch points and/or the like. Basically, any characteristic of the touchevent can affect the topography.

It should be appreciated that the layers 41, 45, 46 can be placed invarious layer stacks and, in some cases, integrated into the same layerstack, depending on the needs of the device. For example, the layers 41,45, 46 can be ordered differently from top to bottom. In one example,the shape changeable layer can be disposed above the sensing and displaylayers. In another example, the sensing layer can be position above theshape changeable layer. Alternatively, the sensing layer and shapechangeable layer can be integrated into the same layer. In anotherexample, the shape changeable layer can be disposed below the displayand touch sensing layer. In yet another example, all the layers can beintegrated. In yet another example, the display and sensing can beintegrated into the same layer.

FIG. 5 illustrates an exemplary method for changing a topography of auser interface according to embodiments of the invention. In the exampleof FIG. 5, a determination can be made about whether an input has beenreceived (51). The input can be from a user of a touch sensitive device.For example, the user can input a request to form a particular userinterface or can touch or near touch the user interface surface. Theinput can also be from an application running on the device. Forexample, a telephone application running on the device can input acommand to form a telephone user interface. The input can also be fromthe device itself. For example, upon powering up, a device can input acommand to form a user interface for that particular device type.

Based on the input, a user interface state can be obtained (53). Theuser interface can be configured based on the obtained user interfacestate (55). The configuring can include at least a physical change ofthe user interface. The user interface can be in communication with thehost system or device, which can drive the configuring. Alternatively,the user interface can frequently ask for a new configuration.

The configuring can include configuring the topography of the userinterface according to the user interface state. For example, the shapeor contour of the user interface can be adjusted to match preferencesfor the user interface state, where the control shape module can map thecontour to the user interface state, thereby creating variable,different tactile feels. The configuring can also include configuringtouch characteristics of the user interface according to the userinterface state. For example, the null touch areas and the active touchareas can be configured and associated with the shape and contour of theuser interface. The configuring can also include configuring displaycharacteristics of the user interface according to the user interfacestate. For example, user interface elements can be displayed that areassociated with the shape and contour of the user interface.

The configuring can also include configuring the topography of the userinterface dynamically. For example, the shape or contour of the userinterface can be adjusted based on a touch event, thereby dynamicallycreating different tactile feels and feedback.

The regions on the device surface to form the user interface can bedetermined in a number of ways. For example, a look-up table residing inmemory can store, for each user interface state, the list of regionsand/or nodes to be altered to form the user interface. When a userinterface state is obtained, the list of regions and/or nodes can beretrieved from the look-up table.

It is to be understood that the method is not limited to thatillustrated here, but can include additional or other functions forchanging the topography of a user interface according to embodiments ofthe invention.

FIG. 6 illustrates an exemplary touch screen that can change topographyaccording to embodiments of the invention. As shown in FIG. 6, touchscreen 60 can include shape changeable surface 61 having a plurality ofmovable touch screen blocks 62. In some cases, the touch screen blocks62 can be proximate to one another and more particularly adjacent to oneanother. They can be positioned together to form a user interface havinga wide variety of shapes (e.g., square, rectangle, circle, plus, cross,X, concentric, annular, etc.). They can for example be positioned inrows and/or columns in order to form a substantially rectangular userinterface (as shown). Each of the touch screen blocks 62 can include adisplay and touch sensor(s). The touch screen blocks 62 can workindividually or be configured to cooperate with one another to form onelarge touch screen display.

The movement of the touch screen blocks 62 can be widely varied. In oneembodiment, the touch screen blocks 62 can be configured to move up anddown from a nominal position. The nominal position of the blocks 62 canform a tapered, curved or flat surface. In the illustrated embodiment,the nominal position of the blocks 62 can form a substantially flatplanar surface. That is, each of the touch screen blocks can include aflat planar surface, the upper surface of which can be level with theother blocks when in the nominal position. Depending on the desiredneeds of the device, the touch screen blocks can raise, lower and/orremain in the nominal position in order to affect a topography change atthe surface 61. This can be accomplished while adjusting the displayelements being presented and the touch sensors being active on the touchscreen blocks 62. The movement can be analog or binary. In analog, themovement can occur at various distances of up and down, while in binary,the movement can be up and down at a specific distance.

The touch screen blocks 62 can respond to a user's input and/or bechanged by a host system for example in response to a control signal ormode of the host system. In some embodiments, the placement of the touchscreen blocks can be configured to form specific user interfaceelements, as in FIGS. 7 and 8 described below.

FIG. 7 illustrates an exemplary touch screen having a user interfacethat can change topography to form virtual buttons according toembodiments of the invention. In the example of FIG. 7, touch screen 70can have a desired user interface state in which the user interface candisplay two virtual buttons 73 and 74 in the display 79. As such, shapechangeable nodes 72 overlaying the displayed buttons 73 and 74 can beraised on the surface 71, thereby informing the user of the location ofthe buttons to be touched. The underlying display 79 can display “OK”and “Cancel” user interface elements in the display regionscorresponding to the raised nodes. A computing system can have functionsassociated with these displayed elements that can execute when the usertouches the buttons 73 and 74.

FIG. 8 illustrates an exemplary touch screen having a user interfacethat can change topography to form a virtual keypad according toembodiments of the invention. In the example of FIG. 8, touch screen 80can have a desired user interface state in which the user interface candisplay virtual keypad 85 in the display 89. As such, shape changeablenodes 82 overlaying the displayed keypad 85 can be raised on the surface81, thereby informing the user of the locations of the keys in thekeypad to be touched. The underlying display 89 can display the numbersand symbols of the keypad in the display regions corresponding to theraised nodes. A computing system can have functions associated withthese displayed numbers and symbols that can execute when the usertouches the keypad 85.

In the examples of FIGS. 6 through 8, the touch screen blocks can beraised or lowered by individual underlying movable or deformable parts.For example, each touch screen block can include an individual actuatordedicated thereto. Alternatively, a flexible membrane or the shapechangeable membrane with discrete point control can also be used insteadof the individual actuators. The membrane can be placed below the touchscreen blocks to encourage their movement or alternatively above thetouch screen blocks to replace their movement. If the latter, the nodesand membranes can be substantially transparent or semi transparent so asto see the underlying touch displays.

Certain nodes of the user interface can raise above or lower below theinitial surface, some can remain at their previous state above or belowthe initial surface, some can return to the initial surface, and somecan remain unaltered in the initial surface, depending on therequirements for the user interface state. The user interface surfacecan be flat, as illustrated here, curved, or any other suitable shapecapable of changing topography to provide a user interface.

It is to be understood that the shape changeable user interface is notlimited to the user interface states illustrated here, but can includeany state that can provide a user interface. In some instances, multipleuser interface states can be combined.

Furthermore, although not shown, it should be appreciated that atransparent flexible membrane can overlay the touch screen blocks. Whenmoved, the membrane can simply stretch to accommodate the new positionof the touch screen blocks.

Similar to the examples of FIGS. 6 through 8, FIG. 9 illustrates anexemplary touch screen of an electronic device having a user interfacethat can change topography according to embodiments of the invention. Inthe example of FIG. 9, electronic device 90 can include touch screen 95,which can include shape changeable surface 91 having a plurality ofmovable touch screen blocks 92. In this example, the touch screen blocks92 can be sized similarly to a displayed icon 99. That is, the icons 99can coincide with the touch screen blocks 92 so as to provide theability to physically alter a particular icon displayed in the touchscreen block 92. A computing system can have functions associated withthese icons that can execute when the user touches the icon 99. By wayof example, the touch screen 95 can be implemented in an iPhone™manufactured by Apple. Inc of Cupertino, Calif. In this example, therecan exist a matrix of rows and columns that can substantially coincidewith the main page or menu page of the iPhone™. The touch screen can forexample be broken into 4 columns and 5 rows. As shown, the icons 99 andtheir descriptors can substantially fill each of the movable touchscreen blocks 92. The blocks 92 can for example move up and/or downdepending on the needs of the iPhone™ and user. In one example, when auser receives an incoming call, the block 92-a associated with the phoneicon can raise up. The same thing can happen with emails in block 92-bor short message service (SMS) messages in block 92-c. The blocks canalso raise up in accordance with a user touch. For example, as the usermoves a finger around the touch screen, the block that would activateupon lift of finger can raise up, thereby alerting the user as to whichicon is selectable for given touch point. It should be appreciated, thatit is not limited to main page and can be used in all screens of theiPhone™ depending on needs of iPhone™ and user.

As mentioned previously, the touch screen blocks 92 can be raised orlowered by underlying movable or deformable parts. The blocks 92 of theuser interface can raise above or lower below the initial surface, somecan remain at their previous state above or below the initial surface,some can return to the initial surface, and some can remain unaltered inthe initial surface, depending on the requirements for the userinterface state. The nominal user interface surface can be flat, asillustrated here, curved, or any other suitable shape capable ofchanging topography to provide a user interface.

The example shown in FIG. 9 can work particularly well with graphicaluser interfaces that provide a plurality of spaced apart selectableicons oriented in a manner similar to the touch screen blocks 92 (e.g.,matrix or rows and columns). Each of the spaced apart icons can bedisplayed in a separate touch screen block 92. The maximum number oficons that can be displayed can coincide with the total number of touchscreen blocks 62. Of course, the number of icons displayed can be lessthan the maximum as shown in FIG. 9. The number of icons and number oftouch screen blocks can generally depend on the needs of the device.

It is to be understood that the shape changeable user interface is notlimited to the user interface states illustrated here, but can includeany state that can provide a user interface. In some instances, multipleuser interface states can be combined.

FIG. 10 illustrates a side view of an exemplary user interface that canchange topography using electromechanical devices according toembodiments of the invention. In the example of FIG. 10, user interface100 can include dynamic shape changeable surface 101 having individualshape changeable nodes 108. In some embodiments, the dynamic surface101, and more particularly each of the nodes 108 can be made up ofindividual selectable, alterable portions 102 that can receive a touchon a front side. The individual alterable portions 102 can be configuredto change their position or their physical configuration in order toaffect a topography change at the dynamic surface 101.

The individual alterable portions 102 can be widely varied. Theindividual alterable portions can include at least a sensing layer. Thesensing layer can for example be configured to detect the presence of anobject in close proximity thereto and further whether an object istouching. The sensing layer can for example be a touch sensing layer,although it should be appreciated that this is not a limitation and thatother types of sensors can be used. Additionally or alternatively, thealterable portions can include a display element. The display elementcan for example be a display panel. Examples of display elements caninclude LCD, OLED, electronic ink, and the like. The individualalterable portions can include additional elements including for examplecovers, labels, skins and the like.

In one example, the change in the alterable portions 102 can be drivenby shape changeable actuators that form the shape changeable nodes 108.

FIG. 14 illustrates an exemplary electromechanical device that can beused as a shape changeable actuator for changing topography of a userinterface according to embodiments of the invention. In the example ofFIG. 14, actuator 140 (or electromechanical device) can have housing 144having outer surface 141, movable piston 145 within the housing, andsolenoid 143 surrounding the piston that can be electrically stimulatedto actuate the piston. In an unactuated state, the piston 145 can bepartially or fully seated within the housing 144. Whereas in an actuatedstate, the piston 145 can move out of the housing 144 to push againstthe corresponding alterable portion 102 of the node 108 of FIG. 10.Conversely, in an unactuated state, the piston 145 can be partially orfully moved out of the housing 144. Whereas in an actuated state, thepiston 145 can move into the housing 144 to pull away from thecorresponding alterable portion 102 of the node 108 of FIG. 10.Alternatively, in an unactuated state, the piston 145 can be seatedmidway within the housing 144. Whereas, in an actuated state, the piston145 can move further out of the housing 144 to push against thecorresponding alterable portion 102 of the node 108 of FIG. 10 and canmove further into the housing to pull away from the correspondingalterable portion of the node. The amount that the piston 145 moves intoand out of the housing 144 can be controlled by the amount of stimulus,e.g., electrical current, applied to the solenoid 143, according to theuser interface state.

Referring again to FIG. 10, the shape changeable actuators 140 of FIG.14 can be disposed on a back side opposite the front side of thealterable portions 102. In some cases, the alterable portions 102 can bedisposed adjacent to the back side of the surface 101, where eachactuator 140 can be proximate to a particular alterable portion 102 ofthe surface. In some embodiments, the actuators 140 can beelectromechanical devices (symbolically illustrated by the rectangles inFIG. 10 and described in FIG. 14), such as micro actuators,microelectromechanical (MEM) devices, piezoelectronics, and othersuitable such miniature devices and can have various gears, cams, andother electromechanical devices to help actuate the alterable portions102.

In one example, the actuators 140 can have a housing, a movable pistonwithin the housing, and a solenoid surrounding the piston. When anelectrical current passes through the solenoid, the solenoid can producea magnetic field that moves the piston in and out of the housing. Themovement of the piston can for example be used to affect a change of thealterable portion 102.

In some embodiments, each alterable portion 102 can raise above theinitial surface 101 when the corresponding actuator 140 changes to pushagainst the back side of the portion and can lower below the initialsurface when the actuator changes to pull away from the back side of theportion. In some embodiments, where the actuators 140 have “pushagainst” change, each alterable surface portion 102 can raise above theinitial surface 101 when the corresponding actuator 140 pushes againstthe back side of the portion and otherwise remain unaltered in theinitial surface. In some embodiments, where the actuators 140 have “pullaway” change, each alterable surface portion 102 can lower below theinitial surface 101 when the corresponding actuator 140 pulls away fromthe back side of the portion and otherwise remain unaltered in theinitial surface.

In some embodiments (as shown), the alterable portions 102 of thesurface 101 can be configured to form a matrix of rows and columns. Itis to be understood, however, that the surface configuration is not solimited, but can include other suitable configurations. Similarly, insome embodiments (as shown), the actuators 140 can be configured to forma matrix of rows and columns corresponding to the matrix rows andcolumns of the surface's alterable portions 102. It is to be understood,however, that the changeable nodes' configuration is not so limited, butcan include other suitable configurations, including other resolutions.

In some embodiments, the actuators 140 can additionally carry inputsignals between the surface 101 and other components of the userinterface 100. For example, the actuators 140 can carry touch signalsfrom the surface 101 to the device processor to be processed and cancarry display signals from the processor to the surface for display.

The outer side of the alterable portions can be widely varied. They canbe rigid or flexible and in some cases even deformable. They can also beangled, flat or have a curved shape. In general, the outer sides of eachalterable portion can cooperate with the other alterable portions toform a continuous outer side. This continuous outer side can be angled,flat or curved.

FIG. 11 illustrates an exemplary user interface that can changetopography using electromechanical devices to raise portions of the userinterface surface according to embodiments of the invention. In theexample of FIG. 11, user interface 110 can include dynamic shapechangeable surface 111 having individual shape changeable nodes 118. Insome embodiments, the dynamic surface 111, and more particularly each ofthe nodes 118 can be made up of individual selectable, alterableportions 112 that can receive a touch on a front side. The individualalterable portions 112 can be configured to change their position ortheir physical configuration in order to affect a topography change atthe dynamic surface 111. In one example, the change in the alterableportions 112 can be driven by shape changeable actuators that form theshape changeable nodes 118. Shape changeable actuators 140 (which can beelectromechanical devices) of FIG. 14 can be capable of moving thealterable portions 112 between an unactuated state and an actuatedstate. The actuated state can for example include raising and loweringof the alterable portion 112. The motion can be analog or binary. Inanalog, varying heights can be achieved. In binary, the motion can besimply up or down a predetermined distance.

FIG. 12 illustrates an exemplary user interface that can changetopography using electromechanical devices to lower portions of the userinterface surface according to embodiments of the invention. In theexample of FIG. 12, user interface 120 can include dynamic shapechangeable surface 121 having individual shape changeable nodes 128. Insome embodiments, the dynamic surface 121, and more particularly each ofthe nodes 128 can be made up of individual selectable, alterableportions 122 that can receive a touch on a front side. The individualalterable portions 122 can be configured to change their position ortheir physical configuration in order to affect a topography change atthe dynamic surface 121. In one example, the change in the alterableportions 122 can be driven by shape changeable actuators that form theshape changeable nodes 128. Shape changeable actuators 140 (which can beelectromechanical devices) of FIG. 14 can be capable of moving thealterable portions 122 between an unactuated state and an actuatedstate. The actuated state can for example include raising and loweringof the alterable portion 122. The motion can be analog or binary. Inanalog, varying heights can be achieved. In binary, the motion can besimply up or down a predetermined distance.

In a specific example of FIGS. 11 and 12, the actuators 140 of FIG. 14can generally correspond to a solenoid. The solenoid can for exampleinclude a moving piston that can be coupled to the alterable portions.In the example of FIG. 11, the nodes 118 can have an unactuated state inwhich their pistons can be fully seated in their housings. In addition,the nodes 118-a can be actuated such that the piston pushes againsttheir corresponding portions 112-a of the surface 111 and raise thecorresponding portions above the initial surface. The other portions ofthe surface 111 can remain unaltered in the initial surface when theircorresponding nodes 118 are not actuated. In the example of FIG. 12,changeable nodes 128 can have an unactuated state in which their pistonscan be partially moved out of their housings to form the initial surfaceabove the housings. The nodes 128-a can be actuated such that thepistons can pull away from their corresponding portions 122-a of thesurface 121 and lower the corresponding portions below the initialsurface, where the other portions corresponding to unactuated nodes canremain unaltered in the initial surface. Of course, the examples shownin FIGS. 11 and 12 can be combined such that some of the portions can beraised, some of the portions can be lowered and some of the portions canremain unchanged. It should also be appreciated that any number of theportions can remain unchanged or changed depending on the needs of thesystem. That is, the example is not limited to moving pairs of adjacentportions as shown.

In one embodiment, the lower side of the alterable portions can belimited to raising no higher than the upper side of an adjacent portionso as to prevent gaps from forming therebetween. This can for example becontrolled in software. Furthermore, in some cases, the interfacebetween adjacent portions can include an elastic seal which can allowmovement therebetween while preventing a gap from forming. In oneexample, the under side of the portions can include a single seal layerthat can cover the entire surface (excluding the attachment points) andtherefore the interfaces.

In the examples of FIGS. 10 through 12, the nodes 108, 118, 128,respectively, can be proximate to their corresponding dynamic surfaces101, 111, 121 that can be raised or lowered by the underlying nodes. Thedynamic surfaces can be replaced with a flexible membrane or a shapechangeable membrane proximate to the nodes. Alternatively, both thenodes and the dynamic surfaces can be replaced with the shape changeablemembrane that itself can change shape to alter the user interface.

In some embodiments, the dynamic surface can be fixed to unchangeablenodes to prevent either upward or downward altering in those areas. Forexample, some nodes can be unchanged and can have the surface fixed tothese nodes to prevent altering. Unchangeable nodes can be interspersedwith changeable nodes, depending on the needs of the system.

FIG. 13 illustrates an exemplary circuit for changing the user interfacetopography using electromechanical devices according to embodiments ofthe invention. The circuit can for example generally be applied to themechanism shown in FIGS. 10 through 12. The circuit 130 can includeapplication specific integrated circuit (ASIC) 137 that can beoperatively coupled to printed circuit board (PCB) 136 that can beoperatively coupled to a plurality of the changeable nodes (oractuators) 108, 118, 120 of FIGS. 10 through 12, respectively. In FIG.13, the PCB 136 and/or the ASIC 137 can cause a stimulus, e.g., anelectrical current, to be applied to the nodes 139. The nodes 139 caninclude housing 134, movable piston 135, and a solenoid that can beactuated by the stimulus to move the piston out of the housing. Thepistons 135 of the unstimulated nodes 139 can remain stationary. Theconnections between the PCB 136 and/or ASIC 137 and the nodes 139 canhave, for example, individual switches for each node, where a particularnode's switch can close when the node is selected to be actuated so asto transmit the stimulus and can remain open when not. The PCB 136 andthe ASIC 137 can include one or more processors and memory for executingsoftware and/or firmware to change the topography of the user interfaceaccording to embodiments of the invention.

In some embodiments, the user interface can be incorporated into a touchsensitive device. It is to be understood, however, that the userinterface can be incorporated into any device capable of changing thetopography of the user interface.

FIG. 15 illustrates an exemplary user interface of a touch screen thatcan change topography according to embodiments of the invention. Thisembodiment can generally correspond to the embodiment shown in FIGS. 10through 12. In the example of FIG. 15, touch screen 150 can have nodes158 that can change the topography of the touch screen surface 151. Thenodes 158 can for example include alterable portions 152, which caninclude a touch sensing layer and a display layer. The alterableportions 152 can be configured to raise and lower via an actuator, e.g.,an electromechanical device, forming the nodes 158. The actuators canraise or lower the portions 152 in order to form raised and/or lowerregions of the surface. These regions can correspond to a virtualbutton, e.g., an “OK” button being displayed on the touch screen. Assuch, the user can be provided a physical way for identifying thelocation of the virtual button, and more particularly the region foractivating the virtual button. For example, the user can identify theboundary by feeling a raised or lowered edge. As shown, portions 152-aand corresponding nodes 158 can be located where the virtual buttoncould be (symbolically illustrated by “X” in the figure). A crosssection at line A-A shows the nodes 158 and the surface portions 152 inthe touch screen.

FIG. 16 illustrates an exemplary user interface of a touch screen thatcan change topography to form a virtual button according to embodimentsof the invention. In the example of FIG. 16, touch screen 160 can formvirtual push button 163. The cross section at line A-A shows nodes 168-athat can raise alterable portions 162-a of user interface surface 161,thereby forming the push button and informing the user of the locationof the push button to be touched. Display nodes (coinciding withalterable portions 162-a) can display the user interface element “OK” toindicate the push button's function. In some cases, the actuators, e.g.,electromechanical devices, forming the nodes 168-a can be configured toprovide feedback during a push button event. For example, the actuatorscan be configured to provide a vibration or click. They can also providea bias force, which the user can press against in order to activate thebutton.

FIG. 17 illustrates an exemplary method for changing the topography of auser interface of a touch screen according to embodiments of theinvention. In the example of FIG. 17, a determination can be made aboutwhether an input has been received (171). The input can be from a userof a touch sensitive device. For example, the user can input a requestto form a particular user interface. The input can also be a touch ornear touch on the user interface surface. The input can also be from anapplication running on the device. For example, a telephone applicationrunning on the device can input a command to form a telephone userinterface. The input can also be from the device itself. For example,upon powering up, a device can input a command to form a user interfacefor that particular device type.

Based on the input, a user interface state can be obtained for a userinterface surface having a plurality of nodes with at least display,touch sensing, and shape changeable capabilities (172). For example, ifa user inputs a request for a keypad, the user interface state can beobtained that indicates a user interface with a keypad should be formed.If a scroll wheel application starts running, the user interface statecan be obtained that indicates a user interface with a scroll wheelshould be formed. If the device is turned on as a media player, the userinterface state can be obtained that indicates a user interface withaudio buttons should be formed.

According to the user interface state, a selectable user interfaceelement can be displayed on one or more of the nodes (173). Concurrenttherewith, the shape of the one or more nodes associated with the userinterface element can be physically altered (174). For example, for akeypad state, the user interface nodes can raise above the initial userinterface surface to form push buttons and the numbers 0-9 can bedisplayed on the push buttons to be selected by the user. The one ormore nodes can be monitored for a touch or near touch as an input tocause the device to perform an operation associated with the nodes(175).

FIG. 18 illustrates an exemplary touch sensing device that can changetopography according to embodiments of the invention. The touch sensingdevice 180 can generally be configured to sense touches or near touchesabout a surface in order to provide inputs to a host device. The inputscan for example be used to operate a graphical user interface presentedon display 185 of a host device. The touch sensing device 180 caninclude a touch pad, for example. The touch pad can be integrated withinthe host device as shown. Alternatively, the touch pad can be a separatemechanism in communication with the host device. When integrated, thetouch sensing device 180 can be mounted within a housing of the hostdevice or in some cases can be provided integrally with a portion of thehost device as for example the housing or outer layer (e.g., skin) ofthe host device. The host device can be widely varied, and can generallycorrespond to any consumer electronic product. In some embodiments, thetouch pad can be suitable for use in portable electronic devices andmore particularly handheld electronic devices such as mice, cell phonesand media players. In one example, the touch pad can be included in anyof those media players manufactured by Apple Inc. of Cupertino Calif.(e.g., iPod™).

In the example of FIG. 18, the touch sensing device 180 can includeshape changeable surface 181 having a plurality of shape changeablenodes 182. The shape changeable nodes can be configured to be deformablepoints or regions of the surface in order to affect the topographychange at the surface. As shown in FIG. 18, the touch sensing device caninclude an upper unbroken, continuous surface, with no cracks or spacesbetween nodes. The nodes 182 can be integrated into the surface. Assuch, the nodes 182 can be hidden from view (as shown by the dottedlines). The nodes can be arranged in a variety of orientations. In theillustrated embodiment, the nodes can be positioned in a matrix of rowsand columns. Each node can form a pixel capable of altering the surfaceof the touch sensing device in a non-trivial manner. The nodes can beactivated singularly or in combination to form a variety ofthree-dimensional shapes about the surface. The arrangement of thesurface can be flat (as shown) or angled, curved, or otherwise formed.In essence, the state of the surface can take any form. The nodes can beconfigured in a variety of ways. In one example, the surface can includea deformable layer that can be deformed by a plurality of actuatorsforming the nodes. The actuators can be separate from or integrated withthe deformable layer. In some cases, the deformable layer and theactuators can be all in one (e.g., same material).

In one embodiment, various user interface states can be created byadjusting the topography of the touch sensing device surface. This canbe for example accomplished by deforming some areas while leaving otherareas undeformed (e.g., selectively deforming the touch surface).Generally speaking, raised and lowered portions can be created amonglevel areas, thereby creating different physical layouts that canprovide variable tactile feels and looks at the touch surface.

FIG. 19 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel according toembodiments of the invention. In the example of FIG. 19, touch sensingdevice 190 can have a desired user interface state in which the userinterface can form a scroll wheel. As such, shape changeable nodes (suchas nodes 182 of FIG. 18) located where the scroll wheel should be can beraised and/or lowered on the surface 191, thereby informing the user ofthe location of the scroll wheel to be touched. The shape changeablenodes forming center button 193-b can be raised and shape changeablenodes forming surrounding circle 193-a can be lowered. In someembodiments, the shape changeable nodes forming the boundaries of thescroll wheel outer and inner circles can be raised to inform the userthat the outer and inner circle areas can be active for touching. Insome embodiments, the shape changeable nodes forming the interior of theinner circle can be raised to one level and the remaining interior ofthe outer circle can be raised to another level to inform the user thatthese raised areas can be active for touching. In some embodiments,touch sensing pixels corresponding to the unaltered areas of thesurface, e.g., the nodes outside the outer circle, can be deactivated toform a null touch region of the touch sensing device or, conversely,only the touch sensing pixels corresponding to the altered areas of thesurface can be activated to form an active touch region of the touchsensing device. A computing system can have functions associated withthe scroll wheel that can execute when the user touches the scrollwheel. In some embodiments, the user's touch can be a slide, rotate,press, translate, tap, or the like motion.

FIG. 20 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a keypad according toembodiments of the invention. In the example of FIG. 20, touch sensingdevice 200 can have a desired user interface state in which the userinterface can form a keypad. As such, shape changeable nodes (such asnodes 182 of FIG. 18) located where the keypad should be can be raisedand/or lowered on the surface 201, thereby informing the user of thelocation of the keypad to be touched. The shape changeable nodes formingboundaries 204-b of the keys can be raised and shape changeable nodesforming keys 204-a can be unaltered, resulting in recesses for the keys.In some embodiments, the shape changeable nodes can have adjustablecharacter identifiers, described below, to display numbering on thekeypad. In some embodiments, the shape changeable nodes forming theboundaries of the keys can be raised to inform the user where the keysare for touching. Alternatively, the keys themselves can be raised. Acomputing system can have functions associated with the keypad that canexecute when the user touches the keys of the keypad.

FIG. 21 illustrates an exemplary touch sensing device having a userinterface that can change topography to form push buttons according toembodiments of the invention. In the example of FIG. 21, touch sensingdevice 210 can have a desired user interface state in which the userinterface can form a plurality of push buttons 214-a. As such, shapechangeable nodes (such as nodes 182 of FIG. 18) located where the pushbuttons 214-a should be can be raised and/or lowered on the surface 211,thereby informing the user of the location of the push buttons to betouched. The shape changeable nodes forming push buttons 214-a can beraised. The remaining nodes can be unaltered and/or deactivated and theunderlying touch sensors can be deactivated to provide null areas 214-bthat do not respond to touch or change shape. Alternatively, the pushbuttons 214-a can be recessed. A computing system can have functionsassociated with the push buttons 214-a that can execute when the usertouches the push buttons.

FIG. 22 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel according toembodiments of the invention. In the example of FIG. 22, touch sensingdevice 220 can have a desired user interface state in which the userinterface can form a scroll wheel. As such, shape changeable nodes (suchas nodes 182 of FIG. 18) located where the scroll wheel should be can beraised and/or lowered on the surface 221, thereby informing the user ofthe location of the scroll wheel to be touched. The shape changeablenodes forming center button 226-c can be raised to form a dome and theshape changeable nodes forming the outer boundary 226-a can be raised toform a rounded border. The shape changeable nodes forming the scrollwheel circle 226-b can remain unaltered. Similar other embodiments tothose of FIG. 19 can apply here. In some cases, a lowered portion can beused to create separate portions of the rounded border with eachseparate section offering a different functionality (e.g., button). Acomputing system can have functions associated with the scroll wheelthat can execute when the user touches the scroll wheel. In someembodiments, the user's touch can be a slide, rotate, press, translate,tap, or the like motion.

FIG. 23 illustrates an exemplary touch sensing device having a userinterface that can change topography to form a scroll wheel and pushbuttons according to embodiments of the invention. In the example ofFIG. 23, touch sensing device 230 can have a desired user interfacestate in which the user interface can form push buttons 237 and scrollwheel 238. As such, shape changeable nodes (such as nodes 182 of FIG.18) located where the push buttons 237 and the scroll wheel 238 shouldbe can be raised and/or lowered on the surface 231, thereby informingthe user of the location of the push buttons and scroll wheel to betouched. The shape changeable nodes forming push buttons 237 can belowered to form recesses. The shape changeable nodes forming the centerbutton boundary and the outer boundary of scroll wheel 238 can belowered to form recessed channels. Similar other embodiments to those ofFIG. 19 can apply here. A computing system can have functions associatedwith the push buttons 237 and the scroll wheel 238 that can execute whenthe user touches the push buttons or the scroll wheel. In someembodiments, the user's touch on the scroll wheel 238 can be a slide,rotate, press, translate, tap, or the like motion.

In some embodiments, each shape changeable node can include anadjustable character identifier that can change according to the userinterface state. For example, in a phone state, as in FIG. 20, the shapechangeable nodes can create the keypad and the adjustable identifierscan produce the numbering for each key. In a media player state, theshape changeable nodes can create a navigation pad and the adjustableidentifiers can produce the control characters, such as “menu,”“play/pause,” and the like. This can for example be accomplished withillumination, mini displays, and the like. Adjustable characteridentifiers are disclosed in U.S. patent application Ser. No.10/722,948, entitled “Touch Pad for Handheld Device,” and Ser. No.11/591,752, entitled “Touch Pad with Symbols Based on Mode,” thecontents of which are incorporated herein by reference in their entiretyfor all purposes.

In the examples of FIGS. 18 through 23, the shape changeable nodes canhave a flexible membrane that can be stretched, retracted, or otherwiseflexed by underlying movable or deformable parts. The individual nodesor the shape changeable membrane can also be used instead of theflexible membrane.

As shown in FIGS. 18 through 23, certain nodes of the user interface canraise above or lower below the initial surface, some can remain at theirprevious state above or below the initial surface, some can return tothe initial surface, and some can remain unaltered in the initialsurface, depending on the requirements for the user interface state. Theuser interface surface can be flat, as illustrated here, curved, or anyother suitable shape capable of changing topography to provide a userinterface.

The user interface states can be based on a mode or operational state ofa host device. This can be especially helpful in multifunctionaldevices, i.e., devices that incorporate separate function into a singledevice. For example, devices that include phone and music playerfunctionality can utilize the embodiment shown in FIG. 20 when in aphone mode and can utilize the embodiment shown in FIG. 19 when in amusic player mode.

It is to be understood that the shape changeable user interface is notlimited to the user interface states illustrated here, but can includeany state that can provide a user interface. In some instances, multipleuser interface states can be combined.

FIG. 24 illustrates a side view of an exemplary user interface that canchange topography using shape changeable nodes according to embodimentsof the invention. In the example of FIG. 24, user interface 240 caninclude a plurality of selectable shape changeable nodes 248 spreadabout a surface 241. The surface 241 can be made of a membrane offlexible or deformable material, e.g., elastic, silicone, soft plastic,or materials that can temporarily deform into a discrete shape or formbefore returning to its original shape or form, with a front side thatcan receive a touch and a back side opposite the front side that canreceive change from the nodes 248. The nodes 248 can be disposedadjacent to the back side of the surface 241, where each node 248 can beproximate to a particular deformable region of the surface. In somecases, the nodes 248 can be coupled, while in other cases they can beseparate from the membrane.

In one example, the change in the surface 241 can be driven by shapechangeable actuators that form the shape changeable nodes 248.

In some embodiments, the actuators can be made up of shape changeablematerial (symbolically illustrated by the circles in FIG. 24), such asnitinol, piezocrystals, and other suitable such material, that canelongate, shrink, or rotate to change shape. Generally speaking, shapechangeable materials change their shape upon a stimulus. The stimuluscan be electrical current, heat, magnetic, light, pressure, and thelike. In one example, the shape changeable material can have atransformation temperature at which the material can transform from apredetermined original shape into a predetermined new shape. Thetransformation temperature can generally be above room temperature suchthat, upon heating, the material can transform into the new shape.Heating can be supplied by electrical current. When the electricalcurrent passes through the material, the current can generate enoughheat to heat the material to the transformation temperature, therebycausing the transformation. Conversely, when the electrical current isterminated, the material can cool to room temperature and transform backto its original shape. Alternative stimuli, e.g., magnetic energy,light, heat, etc., can also be used to heat the shape changeablematerial. This stimuli can also be used directly instead of heating.

FIG. 28 illustrates an exemplary shape changeable node that can be usedas a shape changeable actuator for changing topography of a userinterface according to embodiments of the invention. In the example ofFIG. 28, actuator 288 (or shape changeable node) can have an initialunstimulated shape. In a stimulated state, the actuator 288 canelongate, stretch, raise, extend, or the like (see actuator 288-a) topush against the corresponding deformable region of the surface 241 ofFIG. 24 to deform that region into a discrete shape or form above theinitial surface (e.g., stretches the surface). Here, the shapechangeable material of the actuators can be formulated to have twoshapes—an initial shape and an elongated shape. During stimulation, anelectrical current, heat or other stimulus can be applied to theactuator such that the actuator elongates to form the actuator 288-a inan elongated shape. Similarly, in a stimulated state, the actuator 288can shorten, retract, lower, retreat, or the like (see actuator 288-b)to pull away from the corresponding deformable region of the surface 241of FIG. 24 to deform that region into a discrete shape or form below theinitial surface. Here, the shape changeable material of the actuator 288can be formulated to have two shapes—an initial shape and a shortenedshape. During stimulation, an electrical current, heat or other stimulican be applied to the actuator 288 such that the actuator shortens toform the actuator 288-b in a shortened shape. In some embodiments wherethe shape changeable material of the actuator can be formulated to havethree shapes—an initial shape, an elongated shape, and a shortenedshape, the actuator can have different stimuli at which the respectivetransformations occur. Alternatively, the actuator can have the samestimulus applied in different amounts to produce the respectivetransformations.

FIG. 29 illustrates another exemplary shape changeable node that can beused as a shape changeable actuator for changing topography of a userinterface according to embodiments of the invention. In the example ofFIG. 29, actuator 298 (or shape changeable node) can have an initialunstimulated shape. In a stimulated state, the actuator 298 can shift,rotate, tilt, or the like to push against the corresponding deformableregion of the surface 241 of FIG. 24 to deform that region into adiscrete shape or form above the initial surface. Here, the shapechangeable material of the actuator 298 can be formulated to have twoshapes—an initial shape and an upright shape. During stimulation, astimulus can be applied to the actuator such that the actuator canshift, rotate, tilt, or the like to form state 298-a. The amount thatthe actuator 298 changes shape can be controlled by the amount ofstimulus, e.g., electrical current, applied to the actuator according tothe user interface state.

Referring again to FIG. 24, in some embodiments, each deformable regionof the surface 241 can deform into a discrete shape or form above theinitial surface when the region's corresponding actuator 248 is selectedto provide a force to push against the back side of the region and candeform into a discrete shape or form below the initial surface when theactuator provides a force to pull away from the back side of the region.In some embodiments, where the actuators 248 have “push against” change,each deformable region can deform into a discrete shape or form abovethe initial surface 241 when the region's corresponding actuator pushesagainst the back side of the region and otherwise remain undeformed inthe initial surface. In some embodiments, where the actuators 248 have“pull away” change, each deformable region can deform into a discreteshape or form below the initial surface 241 when the region'scorresponding actuator pulls away from the back side of the region andotherwise remain undeformed in the initial surface.

In some embodiments (as shown), the alterable regions of the surface 241can be configured to form a matrix grid of rows and columns. It is to beunderstood, however, that the surface configuration is not so limited,but can include other suitable configurations. Similarly, in someembodiments (as shown), the changeable actuators 248 can be configuredto form a matrix of rows and columns corresponding to the matrix grid ofthe surface's alterable regions. It is to be understood, however, thatthe actuators' configuration is not so limited, but can include othersuitable configurations, including other resolutions.

In some embodiments, the actuators or nodes 248 can additionally carryinput signals between the surface 241 and other components of the userinterface 240. For example, the nodes 248 can carry touch signals fromthe surface 241 to the device processor to be processed and can carrydisplay signals from the processor to the surface for display.

In alternate embodiments, the shape changeable actuators 248 can beelectromechanical devices such as micro actuators,microelectromechanical (MEM) devices, piezoelectronics, and othersuitable such miniature devices and can have various gears, cams, andother electromechanical devices to help deform the surface.

FIG. 25 illustrates an exemplary user interface that can changetopography using shape changeable nodes to raise portions of the userinterface surface according to embodiments of the invention. In theexample of FIG. 25, one or more shape changeable nodes 258-a can beselectively stimulated to change shape to push against theircorresponding regions 251-a of the surface 251 and deform thecorresponding regions into discrete arcs above the initial surface. Theother regions of the surface 251 can remain undeformed in the initialsurface when their nodes 258 are not stimulated to change shape.

FIG. 26 illustrates an exemplary user interface that can changetopography using shape changeable nodes to lower portions of the userinterface surface according to embodiments of the invention. In theexample of FIG. 26, one or more shape changeable nodes 268-a can beselectively stimulated to change shape to pull away from theircorresponding regions 261-a of the surface 261 and deform thecorresponding regions into discrete hollows below the initial surface,where the other regions corresponding to unstimulated nodes can remainundeformed in the initial surface.

FIG. 27 illustrates an exemplary circuit for changing the user interfacetopography using shape changeable nodes according to embodiments of theinvention. The circuit can for example generally be used with the userinterface shown in FIGS. 24 through 26. The circuit 270 can include ASIC277 that can be operatively coupled to PCB 276 that can be operativelycoupled to a plurality of the shape changeable nodes 248, 258, 268 ofFIGS. 24 through 26, respectively. In FIG. 27, the PCB 276 and/or ASIC277 can cause a stimulus, e.g., an electrical current, to be applied tothe nodes 278. The nodes 278 can change shape when stimulated by thestimulus and can remain unchanged when not. The connections between thePCB 276 and/or ASIC 277 and the nodes 278 can include, for example,individual switches for each nodes, where a particular node's switch canclose when the node is selected to be stimulated so as to transmit thestimulus and can remain open when not.

In the examples of FIGS. 24 through 26, the shape changeable nodes 248,258, 268, respectively, can be proximate to their respective dynamicsurfaces 241, 251, 261 that can be stretched, retracted, or otherwiseflexed by the underlying nodes. The dynamic surfaces can be replacedwith individual nodes or with a shape changeable membrane proximate tothe nodes. Alternatively, both the nodes and the dynamic surfaces can bereplaced with the shape changeable membrane that itself can change shapeto alter the user interface.

In some embodiments, portions of the dynamic surfaces can be fixed oranchored to prevent either upward or downward deformation in thoseareas. These portions can be interspersed with and around nodes,depending on the needs of the system.

In order to generate inputs at the user interface of FIGS. 24 through26, the user interface can include a sensing device capable of detectinga touch or near touch at the user interface surface. FIGS. 30 through 32illustrate exemplary user interfaces that can change topography havingsensing devices according to embodiments of the invention. In theexample of FIG. 30, user interface 300 can include surface 301 andsensing device 307 integrated within the surface. In the example of FIG.31, user interface 310 can include surface 311 and sensing device 317applied to the surface as a coating or layer. In the example of FIG. 32,user interface 320 can include surface 321 and sensing device 327separated from the surface by distance d, which can be any distancesuitable for operating the user interface according to embodiments ofthe invention. In one example, the sensing device can be a touch sensingdevice. If applied to or integrated within the user interface surface,the sensors of the touch sensing device can be configured to beflexible. The sensors can be embodied as lines (drive lines, senselines, or electrodes). These sensors can be coated onto the surface ofthe membrane. These sensors can also be disposed inside the membrane.The lines and/or electrodes can be physical traces or wires, can beincorporated in a flex circuit, or can be deposited.

FIG. 33 illustrates an exemplary user interface that can changetopography by selectively altering shape changeable nodes according toembodiments of the invention. The user interface can for examplecorrespond to the user interface shown in FIGS. 24 through 32. In theexample of FIG. 33, nodes 338 can be selectively controlled to form aphysical button or key at deformable membrane 331 forming the userinterface surface. This can be accomplished by activating a first set ofnodes 338-a (symbolically illustrated by circles with “X” thereon),while leaving a second set of nodes 338-b unactivated. The first set ofnodes 338-a can create a raised area in the deformable membrane 331. Theraised section can represent the area to be pressed in order to actuatethe button or key (i.e., the raised section can flex under force of afinger pressing thereon). More particularly, the first set of nodes338-a can form outer nodes while the second set of nodes 338-b can forminner nodes. The nodes 338-a of the first set can be stimulated tochange shape, while the node(s) 338-b of the second set can remainunchanged, thereby leaving a pliable portion of the surface within thearea formed by the activated first set of nodes 338-a and above thesecond set of nodes 338-b within that area.

The raised area can form the physical button. Because the outer nodes338-a can span an extended area, the deformable membrane 331 can includea plateau section stretched across the outer nodes. As shown, one ormore inner nodes 338-b can be disposed under the plateau section.

FIG. 34 illustrates an exemplary user interface that can changetopography by selectively pressing on shape changeable nodes accordingto embodiments of the invention. In the example of FIG. 34, userinterface 340 can include a raised section of deformable membrane 341that forms the user interface surface. The raised section can be formedby the activation of a first set of nodes 348-a (symbolicallyillustrated by circles with “X” therein) to form a plateau sectionwithin raised section and above a second set of nodes 348-b that can beunactivated. The space provided between the plateau second and secondset of nodes 348-b can allow a user to press finger 346 against and movethe flexible plateau section to be in close proximity to or in contactwith the second set of nodes 348-b.

FIG. 35 illustrates an exemplary user interface that can changetopography by selectively altering shape changeable nodes in response topressure thereon according to embodiments of the invention. In theexample of FIG. 35, user interface 350 can detect a touch by finger 356on a raised section of deformable membrane 351 forming the userinterface surface and register the touch as a button input. The touchcan be a press on a unactivated node (such as node 348-b of FIG. 34).The pressure of the finger 356 on the unactivated node can stimulate thenode 358-b, causing it to change shape and provide an upward force onthe surface 351. That upward force can be a form of haptic feedback feltby the finger 356 to inform the user that the user's finger did indeedpress on the user interface 350. Other forms of haptic feedback can alsobe provided by the node, e.g., a vibration, an electrical impulse, andthe like. The node 358-b can simply change from an unactivated state toan activated state, thereby helping to lift the finger 356 up. This cangive the user physical feedback that the button has been activated. Theraised section can now be formed by the activation of nodes 358-a, 358-b(symbolically illustrated by circles with “X” therein).

In the examples of FIGS. 33 through 35, the shape changeable nodes 338,348, 358 can be proximate to their respective dynamic surfaces 331, 341,351 that can be stretched, retracted, or otherwise flexed by theunderlying nodes. The dynamic surfaces can be replaced with individualnodes or with a shape changeable membrane proximate to the shapechangeable nodes. Alternatively, both the shape changeable nodes and thedynamic surfaces can be replaced with the shape changeable membrane thatitself changes shape to alter the user interface.

FIG. 36 illustrates an exemplary touch pad having a user interface thatcan sense a touch or near touch and change topography according toembodiments of the invention. In the example of FIG. 36, at least aportion of shape changeable surface 361 can form a scroll wheel havingcenter region 364, outer region 363, and boundaries 362-a and 362-b thatcan physically help define the regions. The shape changeable nodesforming boundaries 362-a and 362-b can be raised and the nodes formingcenter region 364 and outer region 363 can remain unaltered, as shown bya cross section at line A-A. Because the scroll wheel at the centerbutton 364 or within the outer circle 363 can be associated with inputs,these elements can have active touch regions. A touch can be detected inthese regions. By way of example, the center region 364 can represent acenter button used for making selection and initiating commands, i.e.,by pressing, while the outer region 363 can represent a navigationregion for traversing content on a screen, i.e., by swirling a fingerabout the region. Conversely, because the remaining portions of theshape changeable surface 361, i.e., outside the outer boundary 362-b canbe unassociated with inputs, these portions can have null touch regions.A touch can be ignored in these regions. In some embodiments, shapechangeable nodes in these null touch regions can also be deactivated soas not to alter during this user interface state. A computing system canhave functions associated with the active touch regions that can executewhen the user touches these regions. Because the sensing and shapechangeable regions can be dynamic, the shape changeable (protruding,recessed, and flat) and sensed regions (active and null) can be changedfrom what is shown. That is, the computing system can change shape andsensing capabilities while matching them to different functions, thus,enabling switching from the scroll wheel shown to something different,such as a keypad of a phone.

FIGS. 37 and 38 illustrate exemplary touch pads having a user interfacethat can sense a touch or near touch and change topography according toembodiments of the invention. In the example of FIG. 37, touch pad 370can have a series of nodes 372 that can change the shape of surface 371.A cross section at line A-A shows the nodes 372 in the touch pad 370.The touch pad 370 can form a flat surface. In the example of FIG. 38,touch pad 380 can similarly have a series of nodes 382 that can changethe shape of surface 381 to form circular scroll wheel 384. The scrollwheel can be formed by selected nodes (indicated by “X” in FIG. 37, forexample). The cross section at line A-A shows the nodes 382-a that canchange their state in order to raise user interface surface 381 inselect areas or points, thereby forming the scroll wheel 384 andinforming the user of the location of the scroll wheel. In one example,the nodes can include an actuator that can push upwards on the innersurface of an elastic or flexible member, thereby raising a selectportion of the surface. The actuators can be widely varied. Theactuators can for example be formed from a shape changeable material. Asmentioned above in FIG. 36, a touch surface can be controlled with nulland activate regions as needed.

FIG. 39 illustrates an exemplary method of changing the topography of auser interface of a touch pad according to embodiments of the invention.The method can for example be used in any of the examples shown in FIGS.24 through 38. In the example of FIG. 39, a determination can be madeabout whether an input has been received (391). The input can be from auser of a touch sensitive device. For example, the user can input arequest to form a particular user interface. The input can also be atouch or near touch on the user interface surface. The input can also befrom an application running on the device. For example, a telephoneapplication running on the device can input a command to form atelephone user interface. The input can also be from the device itself.For example, upon powering up, a device can input a command to form auser interface for that particular device type.

Based on the input, a user interface state can be obtained for a userinterface surface having a plurality of nodes with at least touchsensing and shape changeable capabilities (393). For example, if a userinputs a request for a keypad, the user interface state can be obtainedthat indicates a user interface with a keypad should be formed. If ascroll wheel application starts running, the user interface state can beobtained that indicates a user interface with a scroll wheel should beformed. If the device is turned on as a media player, the user interfacestate can be obtained that indicates a user interface with audio buttonsshould be formed.

According to the user interface state, the user interface surface can bephysically altered from a first physical layout to a second physicallayout, where each layout can represent a mode of an electronic device(395). For example, for a keypad state, the user interface surface canphysically alter from a flat layout with no buttons to a non-flat layoutwith numerous keypad buttons to represent a keypad mode of the device.

The surface can be monitored for a touch or near touch as an input tocause the device to perform an operation (397). In some cases, the touchsensors can be monitored at select locations, i.e., areas assigned toreceive touch inputs in order to initiate a certain functionality. Byway of example, the monitoring circuit can create active and nullregions.

FIG. 40 illustrates a side view of an exemplary user interface that canchange topography using a shape changeable membrane according toembodiments of the invention. In the example of FIG. 40, user interface400 can include dynamic surface 401. The dynamic surface 401 can be alayer or a matrix of shape changeable material that can elongate,shrink, or rotate to change shape when selectively stimulated. Theamount that the surface 401 changes shape can be controlled by theamount of stimulus, e.g., electrical current, applied to the surface,according to the user interface state of the surface. Any suitable shapechangeable material can be used, including for example nitinol. Theshape changeable material can be a single deformable membrane or it canfor example work in cooperation with a deformable or flexible material(e.g., embedded or applied to a surface).

In some embodiments, the alterable regions of the surface 401 can beconfigured to form a matrix grid of rows and columns. It is to beunderstood, however, that the surface configuration is not so limited,but can include other suitable configurations.

In some embodiments, a particular region of the surface 401 can changeshape to form a discrete shape or form above the initial surface whenthe region is stimulated and can change shape to form a discrete shapeor form below the initial surface when the region is stimulated.

FIG. 41 illustrates a side view of an exemplary user interface that canchange topography using a shape changeable membrane to raise portions ofthe user interface surface according to embodiments of the invention. Inthe example of FIG. 41, shape changeable regions 412-a can beselectively stimulated to change shape to form a discrete arc above theinitial surface 411. The other regions of the surface 411 can remainunchanged in the initial surface when they are not stimulated.

FIG. 42 illustrates a side view of an exemplary user interface that canchange topography using a shape changeable membrane to lower portions ofthe user interface surface according to embodiments of the invention. Inthe example of FIG. 42, shape changeable regions 422-a can beselectively stimulated to change shape to form a discrete hollow belowthe initial surface 421, while the other regions of the surface canremain unchanged in the initial surface when they are not stimulated.

FIG. 43 illustrates an exemplary circuit for changing the user interfacetopography using a shape changeable membrane according to embodiments ofthe invention. The circuit 430 can include ASIC 437 that can beoperatively coupled to PCB 436 that can be operatively coupled to thedynamic surfaces 401, 411, 421 of FIGS. 40 through 42, respectively. InFIG. 43, the PCB 436 and/or the ASIC 437 can cause a stimulus, e.g., anelectrical current, to be applied to alterable regions of the surface431. The alterable regions can change shape when stimulated by thestimulus and can remain unchanged when not. The connections between thePCB 436 and/or ASIC 437 and the regions can include, for example,individual switches for each region, where a particular region's switchcan close when the region is selected to be stimulated so as to transmitthe stimulus and can remain open when not. Alternatively, the PCB 436can be coupled to the surface via a flex circuit. Alternatively, the PCB436 can be replaced with a flex circuit.

It should be appreciated that the user interface can have touch sensingcapabilities. The sensing technology can be integrated with (e.g.,embedded), applied to (e.g., attached) or be separate from the surface.In one example, capacitive sensing can be used. For example, the sensorscan be embedded or applied to the inner surface of the user interface.In another example, proximity sensing can be used. For example,proximity sensors can be disposed underneath but decoupled from the userinterface surface. Of course, other sensing technologies can also beused.

FIGS. 44 through 48 illustrate exemplary user interfaces that can changetopography based on a location of a touch event according to embodimentsof the invention. Rather than altering shape changeable nodes of a userinterface prior to a user touching the user interface surface, the nodescan be altered dynamically as the user touches or near touches thesurface at a certain user interface element, thereby informing the userof the location of the user interface element being or about to betouched. In the example of FIG. 44, shape changeable nodes 442 of userinterface 440 can be unaltered. In the example of FIG. 45, finger 457can touch a region or point of user interface surface 451, therebycausing node 452-a, associated with that region or point, to change theshape near the region or point. For example, the node can form a raisedor recessed region or point proximate to the touch. The user thereforecan know where the user is touching at any given moment during a touchevent. In the example of FIG. 46, as finger 467 moves to a differentlocation (e.g., makes sliding, scrolling, tapping, or the like motion456 of FIG. 45 to the right) on user interface surface 461, node 462-aproximate to the new location can change the shape of the surface andthe previously touched node (such as node 452-a of FIG. 45) can changeback to its original state. By way of example, if raised, node 452-a canlower back to a nominal position and, if lowered, node 452-a can raiseback to the nominal position. In the example of FIG. 47, as finger 477moves to a different location (e.g., makes sliding, scrolling, tapping,or the like motion 466 of FIG. 46 downward) on user interface surface471, node 472-a proximate to the new location can change the shape ofthe surface and the previously touched node (such as node 462-a of FIG.46) can change back to its original state. By way of example, if raised,node 462-a can lower back to a nominal position and, if lowered, node462-a can raise back to the nominal position. In the example of FIG. 48,multi-touch can be implemented, where multiple fingers 487-a and 487-btouching respective nodes 482-a and 482-b can cause the nodes to changethe shape of user interface surface 481 near each of the touches.

In one embodiment, the user interfaces shown in FIGS. 44 through 48 canbe a single-touch or multi-touch screen display and can include aplurality of touch sensors grouped in rows and columns, for example, orsome other appropriate configuration. The touch sensors can detect oneor more touches or near touches on the user interface surface at one ormore locations. The touch sensors can work as discrete sensors ortogether to form a larger sensor. Each touch sensor can raise, lower, orremain at an initial state, depending on the needs of the device. Thetouch display of the touch screen can also include a plurality of touchdisplay nodes grouped in rows and columns, for example, or some otherappropriate configuration. The display nodes can work as discretedisplays or together to form a larger display (for example, a maindisplay capable of distributing video or other applications). Each touchdisplay node can operate as a shape changeable node by raising,lowering, or remaining at an initial state, individually, sequentially,collectively, or the like in order to change the user interfacetopography depending on the needs of the device.

In the examples of FIGS. 44 through 48, the shape changeable nodes canbe individual nodes that can be raised or lowered by underlying movableor deformable parts. A flexible membrane or a shape changeable membranecan also be used instead of the individual nodes. The nodes andmembranes can be substantially transparent or semi transparent so as tosee the underlying touch displays.

FIG. 49 illustrates an exemplary method for changing the topography of auser interface based on a location of a touch event according toembodiments of the invention. In the example of FIG. 49, a determinationcan be made about whether an input has been received (491). The inputcan be from a user of a touch sensitive device. For example, the usercan input a request to form a particular user interface. The input canalso be a touch or near touch on the user interface surface. The inputcan also be from an application running on the device. For example, atelephone application running on the device can input a command to forma telephone user interface. The input can also be from the deviceitself. For example, upon powering up, a device can input a command toform a user interface for that particular device type.

Based on the input, a user interface state can be obtained for a userinterface surface having a plurality of nodes with at least touchsensing and shape changeable capabilities (493). For example, if a user,an application, or a device inputs a request for a user interfacesurface that physically alters based on a location of a touch or neartouch, the user interface state can be obtained, indicating that a userinterface alterable by a touch event should be formed.

A determination can be made about whether a touch event has beendetected (495). The touch event can be either a touch or a near touch onthe user interface surface.

Upon detection of the touch event and its location on the user interfacesurface, the surface can physically alter at that location from a firstphysical layout to a second physical layout, where each layout canrepresent a mode of an electronic device (497). For example, as a touchor near touch occurs at a location on the surface, that location canraise or lower to form the second physical layout, depending on theneeds of the device. As the touch or near touch moves away from thelocation, that location can return to its original position to form thefirst physical layout, depending on the needs of the device.

FIG. 50 illustrates an exemplary display device having a user interfacethat can change topography according to embodiments of the invention. Inthe example of FIG. 50, display device 500 can include translucentstretchable membrane 503 disposed over display 502, which can be in turndisposed on changeable nodes 501, which can be activated to raise orlower the display.

FIG. 51 illustrates an exemplary display device having a user interfacethat can change topography by raising a display screen according toembodiments of the invention. In the example of FIG. 51, changeablenodes 511 can be stimulated to raise display 512. Upon raising, thedisplay 512 can push against translucent stretchable membrane 513,causing the membrane to stretch and thin out at section 513-a. As themembrane thins out, it can become more transparent, making display 512easier to view.

In some embodiments, the changeable nodes 501, 511 can beelectromechanical devices. In some embodiments, the changeable nodes canbe shape changeable material. In some embodiments, the changeable nodescan be stimulated to a user interface state that can include displayviewing. In some embodiments, the membrane can be somewhat opaque in itsnominal state and translucent or transparent in its stretched state.

FIG. 52 illustrates an exemplary user interface that can changetopography according to embodiments of the invention. In the example ofFIG. 52, user interface 520 can include shape changeable user interfacesurface 521 having transparent or semi transparent flexible outermembrane 523 and shape changeable nodes 522, transparent or semitransparent touch sensing nodes 525, and display nodes 526. The flexibleouter membrane 523 can form the touchable surface of the user interfaceof a touch sensitive device and can be expanded or retracted as theunderlying shape changeable nodes 522 are altered, thereby changing thetopography at the user interface surface 521. The membrane 523 can beelastic, silicone, rubber, soft plastic, or any material that canstretch under force and return to normal when the force is removed.

The touch sensing nodes 525 can be disposed adjacent to the shapechangeable nodes 522 on an undersurface of the membrane 523, where theundersurface can be opposite the touchable surface. The touch sensingnodes 525 can detect a touch or near touch on the surface of themembrane 523. As shown in FIG. 52, the touch sensing nodes 525 can bepositioned alternately with the shape changeable nodes 522.Alternatively, the touch sensing nodes 525 can be positioned betweenevery two or more shape changeable nodes 522 or vice versa, depending onthe needs of the user interface 520.

The display nodes 526 can be disposed adjacent to the shape changeablenodes 522 and the touch sensing nodes 525. The display nodes 526 candisplay user interface elements viewable through the membrane 523. Asshown in FIG. 52, the display nodes 526 can be aligned with the touchsensing nodes 525. Alternatively, the display nodes 526 can be alignedwith two or more touch sensing nodes 525 or vice versa and/or alignedwith shape changeable nodes 522, depending on the needs of the userinterface 520.

In some embodiments, the touch region and the shape changeable regionscan substantially coincide. In some embodiments, the shape changeableregions can be positioned within the touch region, being either the samesize as the touch region, smaller, or larger.

FIG. 53 illustrates an exemplary computing system that can include oneor more of the embodiments of the invention described herein. In theexample of FIG. 53, computing system 530 can include one or more panelprocessors 531 and peripherals 532, and panel subsystem 533. Peripherals532 can include, but are not limited to, random access memory (RAM) orother types of memory or storage, watchdog timers and the like. Panelsubsystem 533 can include, but is not limited to, one or more sensechannels 533-a, channel scan logic (analog or digital) 533-b and driverlogic (analog or digital) 533-c. Channel scan logic 533-b can access RAM533-f, autonomously read data from sense channels 533-a and providecontrol for the sense channels. In addition, channel scan logic 533-bcan control driver logic 533-c to generate stimulation signals 533-d atvarious phases that can be simultaneously applied to drive lines oftouch sensor panel 534. Panel subsystem 533 can operate at a low digitallogic voltage level (e.g. 1.7 to 3.3V). Driver logic 533-c can generatea supply voltage greater that the digital logic level supply voltages bycascading two charge storage devices, e.g., capacitors, together to formcharge pump 533-e. Charge pump 533-e can be used to generate stimulationsignals 533-d that can have amplitudes of about twice the digital logiclevel supply voltages (e.g. 3.4 to 6.6V). Although FIG. 53 shows chargepump 533-e separate from driver logic 533-c, the charge pump can be partof the driver logic. In some embodiments, panel subsystem 533, panelprocessor 531 and peripherals 532 can be integrated into a singleapplication specific integrated circuit (ASIC).

Touch sensor panel 534 can include a capacitive sensing medium having aplurality of drive lines and a plurality of sense lines, although othersensing media can also be used. The drive and sense lines can be formedfrom a transparent conductive medium such as Indium Tin Oxide (ITO) orAntimony Tin Oxide (ATO), although other transparent and non-transparentmaterials such as copper can also be used. The drive and sense lines canbe formed on a single side of a substantially transparent substrate, onopposite sides of the substrate, or on two separate substrates separatedby the dielectric material. Each intersection of drive and sense linescan represent a capacitive sensing node and can be viewed as pictureelement (pixel) 534-a, which can be particularly useful when touchsensor panel 534 is viewed as capturing an “image” of touch. (In otherwords, after panel subsystem 533 has determined whether a touch eventhas been detected at each touch sensor in the touch sensor panel, thepattern of touch sensors in the multi-touch panel at which a touch eventoccurred can be viewed as an “image” of touch (e.g. a pattern of fingerstouching the panel).) The capacitance between the drive and sense linesand local system ground appears as a stray capacitance Cstray and thecapacitance at the intersections of the drive and sense lines, i.e., thepixels, as a mutual signal capacitance Csig when the given drive line isstimulated with an alternating current (AC) signal. The presence of afinger or other object near or on the touch sensor panel can be detectedby measuring changes to a signal charge present at the pixels beingtouched, which is a function of Csig. Each sense line of touch sensorpanel 534 can drive sense channel 533-a in panel subsystem 533.

Dynamic shape change device 535 can change topography of a userinterface of the computing system 530 according to embodiments of theinvention. The shape change device 535 can have movable or deformableregions which can be selected to alter outward or inward to form a userinterface, where the resulting user interface surface can inform theuser of the location of user interface elements to be touched.

Computing system 530 can also include host processor 537 for receivingoutputs from panel processor 531 and performing actions based on theoutputs that can include, but are not limited to, moving one or moreobjects such as a cursor or pointer, scrolling or panning, adjustingcontrol settings, opening a file or document, viewing a menu, making aselection, executing instructions, operating a peripheral device coupledto the host device, answering a telephone call, placing a telephonecall, terminating a telephone call, changing the volume or audiosettings, storing information related to telephone communications suchas addresses, frequently dialed numbers, received calls, missed calls,logging onto a computer or a computer network, permitting authorizedindividuals access to restricted areas of the computer or computernetwork, loading a user profile associated with a user's preferredarrangement of the computer desktop, permitting access to web content,launching a particular program, encrypting or decoding a message, and/orthe like. Host processor 537 can also perform additional functions thatmay not be related to panel processing, and can be coupled to programstorage 536 and display device 538 such as a liquid crystal display(LCD) for providing a user interface to a user of the device. Displaydevice 538 together with touch sensor panel 534, when located partiallyor entirely with the touch sensor panel, can form a touch screen.Dynamic shape change device 535 together with touch sensor panel 534 canprovide a shape changeable user interface.

Note that one or more of the functions described above can be performedby firmware stored in memory (e.g. one of the peripherals 532 in FIG.53) and executed by panel processor 531, or stored in program storage536 and executed by host processor 537. The firmware can also be storedand/or transported within any computer-readable medium for use by or inconnection with an instruction execution system, apparatus, or device,such as a computer-based system, processor-containing system, or othersystem that can fetch the instructions from the instruction executionsystem, apparatus, or device and execute the instructions. In thecontext of this document, a “computer-readable medium” can be any mediumthat can contain or store the program for use by or in connection withthe instruction execution system, apparatus, or device. The computerreadable medium can include, but is not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus or device, a portable computer diskette (magnetic), a randomaccess memory (RAM) (magnetic), a read-only memory (ROM) (magnetic), anerasable programmable read-only memory (EPROM) (magnetic), a portableoptical disc such a CD, CD-R, CD-RW, DVD, DVD-R, or DVD-RW, or flashmemory such as compact flash cards, secured digital cards, universalserial bus (USB) memory devices, memory sticks, and the like.

The firmware can also be propagated within any transport medium for useby or in connection with an instruction execution system, apparatus, ordevice, such as a computer-based system, processor-containing system, orother system that can fetch the instructions from the instructionexecution system, apparatus, or device and execute the instructions. Inthe context of this document, a “transport medium” can be any mediumthat can communicate, propagate or transport the program for use by orin connection with the instruction execution system, apparatus, ordevice. The transport readable medium can include, but is not limitedto, an electronic, magnetic, optical, electromagnetic or infrared wiredor wireless propagation medium.

It is to be understood that the sensor panel is not limited to a touchsensor panel, as described in FIG. 53, but can be a proximity sensorpanel or any other sensor panel capable of sensing a touch or hoverevent and having a user interface to change topography according toembodiments of the invention. Furthermore, although the touch sensors inthe touch sensor panel can be described herein in terms of an orthogonalarray of touch sensors having rows and columns, it should be understoodthat embodiments of this invention are not limited to orthogonal arrays,but can be generally applicable to touch sensors arranged in any numberof dimensions and orientations, including diagonal, concentric circle,and three-dimensional and random orientations. In addition, the touchsensor panel described herein can be either a single-touch or amulti-touch sensor panel.

FIG. 54 illustrates an exemplary mobile telephone 540 that can includetouch sensor panel 544, shape change device 545, display device 543, andother computing system blocks that can be utilized for changingtopography of a user interface of the telephone.

FIG. 55 illustrates an exemplary digital media player 550 that caninclude touch sensor panel 554, display device 553, shape change device555-a which can alter a portion of the media player associated with thetouch sensor panel and the display device, shape change device 555-bwhich can alter a portion of the media player to display scroll wheel556, for example, and other computing system blocks that can be utilizedfor changing topography of a user interface of the media player.

FIG. 56 illustrates an exemplary personal computer 560 that can includetouch sensor panel (trackpad) 564, shape change device 565, display 563,and other computing system blocks that can be utilized for changingtopography of a user interface of the personal computer.

The mobile telephone, media player, and personal computer of FIGS. 54through 56, respectively, can improve the user experience by providing auser interface that can change topography according to embodiments ofthe invention.

Although embodiments of this invention have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of embodiments of this invention as defined bythe appended claims.

What is claimed is:
 1. An electronic device having a touch sensitiveinput surface, comprising: a shape changeable layer; a touch sensinglayer formed along the shape changeable layer; and a controllercommunicatively coupled to the shape changeable layer and the touchsensing layer, the controller capable of receiving one or more firstsignals from the touch sensing layer, tracking movement of an objectproximate the input surface based on the received one or more firstsignals, and transmitting one or more second signals to the shapechangeable layer to dynamically alter a topography of the input surfaceat locations of the input surface corresponding to the tracked movement.2. The electronic device of claim 1, the controller further capable ofdynamically altering the topography of the input surface in accordancewith a particular user interface state detected during the trackedmovement.
 3. The electronic device of claim 1, the controller furthercapable of tracking the movement of the proximate object when the objecttouches and moves across the input surface.
 4. The electronic device ofclaim 1, the controller further capable of tracking the movement of theproximate object when the object hovers and moves over the inputsurface.
 5. The electronic device of claim 1, the controller furthercapable of tracking the movement of the proximate object when the objectpresses on the input surface.
 6. The electronic device of claim 1, thecontroller further capable of dynamically altering the topography of theinput surface based on a location of the tracked movement.
 7. Theelectronic device of claim 1, the controller further capable ofdynamically altering the topography of the input surface based on anacceleration of the tracked movement.
 8. The electronic device of claim1, the controller further capable of dynamically altering the topographyof the input surface based on a direction of the tracked movement. 9.The electronic device of claim 1, the controller further capable ofdynamically altering the topography of the input surface based on a sizeof the proximate object.
 10. A surface for generating haptic feedback,comprising: a shape changeable layer; and a plurality of actuatorscoupled with the shape changeable layer; wherein each of the pluralityof actuators are communicatively couplable to a controller andconfigurable for dynamically altering a topography of the shapechangeable layer at locations of the surface corresponding to an objectmoving proximate to the surface.
 11. The surface of claim 10, at leastone of the plurality of actuators further configurable for generating avibration at the shape changeable layer to alter the topography of theshape changeable layer.
 12. The surface of claim 10, at least one of theplurality of actuators further configurable for altering the topographyof the shape changeable layer in response to an electromagneticstimulus.
 13. The surface of claim 10, at least one of the plurality ofactuators further configurable for generating haptic feedback inresponse to a generated electric field.
 14. The surface of claim 10, atleast one of the plurality of actuators further configurable forgenerating an electrical impulse at the shape changeable layer to alterthe topography of the shape changeable layer.
 15. The surface of claim10, further comprising a touch sensing layer formed along the shapechangeable layer, the touch sensing layer communicatively couplable tothe controller and configurable for sending signals to the controllerindicative of the object moving proximate to the touch surface.
 16. Amethod of providing haptic feedback at a surface, comprising: trackingmovement of an object proximate an input surface; and dynamicallyaltering a topography of an input surface at locations of the inputsurface corresponding to the tracked movement.
 17. The method of claim16, further comprising dynamically altering the topography of the inputsurface in accordance with a particular user interface state detectedduring the tracked movement.
 18. The method of claim 16, whereintracking the movement of the object comprises tracking the movement ofthe proximate object when the object touches and moves across the inputsurface.
 19. The method of claim 16, wherein tracking the movement ofthe object comprises tracking the movement of the proximate object whenthe object hovers and moves over the input surface.
 20. The method ofclaim 16, wherein tracking the movement of the object comprises trackingthe movement of the proximate object when the object presses on theinput surface.
 21. The method of claim 16, further comprisingdynamically altering the topography of the input surface based on alocation of the tracked movement.
 22. The method of claim 16, furthercomprising dynamically altering the topography of the input surfacebased on an acceleration of the tracked movement.
 23. The method ofclaim 16, further comprising dynamically altering the topography of theinput surface based on a direction of the tracked movement.
 24. Themethod of claim 16, further comprising dynamically altering thetopography of the input surface based on a size of the proximate object.